Training MLPs layer-by-layer with the information potential

In the area of information processing one fundamental issue is how to measure the statistical relationship between two variables based only on their samples. In a previous paper, the idea of Information Potential which was formulated from the so called Quadratic Mutual Information was introduced, and successfully applied to problems such as Blind Source Separation and Pose Estimation of SAR (Synthetic Aperture Radar) Images. This paper shows how information potential can be used to train a MLP (multilayer perceptron) layer-by-layer, which provides evidence that the hidden layer of a MLP serves as an “information filter” which tries to best represent the desired output in that layer in the statistical sense of mutual information.